126 research outputs found
FedSEAL: Semi-Supervised Federated Learning with Self-Ensemble Learning and Negative Learning
Federated learning (FL), a popular decentralized and privacy-preserving
machine learning (FL) framework, has received extensive research attention in
recent years. The majority of existing works focus on supervised learning (SL)
problems where it is assumed that clients carry labeled datasets while the
server has no data. However, in realistic scenarios, clients are often unable
to label their data due to the lack of expertise and motivation while the
server may host a small amount of labeled data. How to reasonably utilize the
server labeled data and the clients' unlabeled data is thus of paramount
practical importance. In this paper, we propose a new FL algorithm, called
FedSEAL, to solve this Semi-Supervised Federated Learning (SSFL) problem. Our
algorithm utilizes self-ensemble learning and complementary negative learning
to enhance both the accuracy and the efficiency of clients' unsupervised
learning on unlabeled data, and orchestrates the model training on both the
server side and the clients' side. Our experimental results on Fashion-MNIST
and CIFAR10 datasets in the SSFL setting validate the effectiveness of our
method, which outperforms the state-of-the-art SSFL methods by a large margin.Comment: 15 pages, 7 figure
On the Expected Discounted Penalty Function for the Classical Risk Model with Potentially Delayed Claims and Random Incomes
We focus on the expected discounted penalty function of a compound Poisson risk model with random incomes and potentially delayed claims. It is assumed that each main claim will produce a byclaim with a certain probability and the occurrence of the byclaim may be delayed depending on associated main claim amount. In addition, the premium number process is assumed as a Poisson process. We derive the integral equation satisfied by the expected discounted penalty function. Given that the premium size is exponentially distributed, the explicit expression for the Laplace transform of the expected discounted penalty function is derived. Finally, for the exponential claim sizes, we present the explicit formula for the expected discounted penalty function
Loghub: A Large Collection of System Log Datasets towards Automated Log Analytics
Logs have been widely adopted in software system development and maintenance
because of the rich system runtime information they contain. In recent years,
the increase of software size and complexity leads to the rapid growth of the
volume of logs. To handle these large volumes of logs efficiently and
effectively, a line of research focuses on intelligent log analytics powered by
AI (artificial intelligence) techniques. However, only a small fraction of
these techniques have reached successful deployment in industry because of the
lack of public log datasets and necessary benchmarking upon them. To fill this
significant gap between academia and industry and also facilitate more research
on AI-powered log analytics, we have collected and organized loghub, a large
collection of log datasets. In particular, loghub provides 17 real-world log
datasets collected from a wide range of systems, including distributed systems,
supercomputers, operating systems, mobile systems, server applications, and
standalone software. In this paper, we summarize the statistics of these
datasets, introduce some practical log usage scenarios, and present a case
study on anomaly detection to demonstrate how loghub facilitates the research
and practice in this field. Up to the time of this paper writing, loghub
datasets have been downloaded over 15,000 times by more than 380 organizations
from both industry and academia.Comment: Dateset available at https://zenodo.org/record/322717
Co-evolving Vector Quantization for ID-based Recommendation
Category information plays a crucial role in enhancing the quality and
personalization of recommendations. Nevertheless, the availability of item
category information is not consistently present, particularly in the context
of ID-based recommendations. In this work, we propose an alternative approach
to automatically learn and generate entity (i.e., user and item) categorical
information at different levels of granularity, specifically for ID-based
recommendation. Specifically, we devise a co-evolving vector quantization
framework, namely COVE, which enables the simultaneous learning and refinement
of code representation and entity embedding in an end-to-end manner, starting
from the randomly initialized states. With its high adaptability, COVE can be
easily integrated into existing recommendation models. We validate the
effectiveness of COVE on various recommendation tasks including list
completion, collaborative filtering, and click-through rate prediction, across
different recommendation models. We will publish the code and data for other
researchers to reproduce our work
FinalMLP: An Enhanced Two-Stream MLP Model for CTR Prediction
Click-through rate (CTR) prediction is one of the fundamental tasks for
online advertising and recommendation. While multi-layer perceptron (MLP)
serves as a core component in many deep CTR prediction models, it has been
widely recognized that applying a vanilla MLP network alone is inefficient in
learning multiplicative feature interactions. As such, many two-stream
interaction models (e.g., DeepFM and DCN) have been proposed by integrating an
MLP network with another dedicated network for enhanced CTR prediction. As the
MLP stream learns feature interactions implicitly, existing research focuses
mainly on enhancing explicit feature interactions in the complementary stream.
In contrast, our empirical study shows that a well-tuned two-stream MLP model
that simply combines two MLPs can even achieve surprisingly good performance,
which has never been reported before by existing work. Based on this
observation, we further propose feature gating and interaction aggregation
layers that can be easily plugged to make an enhanced two-stream MLP model,
FinalMLP. In this way, it not only enables differentiated feature inputs but
also effectively fuses stream-level interactions across two streams. Our
evaluation results on four open benchmark datasets as well as an online A/B
test in our industrial system show that FinalMLP achieves better performance
than many sophisticated two-stream CTR models. Our source code will be
available at MindSpore/models.Comment: Accepted by AAAI 2023. Code available at
https://xpai.github.io/FinalML
ReLoop2: Building Self-Adaptive Recommendation Models via Responsive Error Compensation Loop
Industrial recommender systems face the challenge of operating in
non-stationary environments, where data distribution shifts arise from evolving
user behaviors over time. To tackle this challenge, a common approach is to
periodically re-train or incrementally update deployed deep models with newly
observed data, resulting in a continual training process. However, the
conventional learning paradigm of neural networks relies on iterative
gradient-based updates with a small learning rate, making it slow for large
recommendation models to adapt. In this paper, we introduce ReLoop2, a
self-correcting learning loop that facilitates fast model adaptation in online
recommender systems through responsive error compensation. Inspired by the
slow-fast complementary learning system observed in human brains, we propose an
error memory module that directly stores error samples from incoming data
streams. These stored samples are subsequently leveraged to compensate for
model prediction errors during testing, particularly under distribution shifts.
The error memory module is designed with fast access capabilities and undergoes
continual refreshing with newly observed data samples during the model serving
phase to support fast model adaptation. We evaluate the effectiveness of
ReLoop2 on three open benchmark datasets as well as a real-world production
dataset. The results demonstrate the potential of ReLoop2 in enhancing the
responsiveness and adaptiveness of recommender systems operating in
non-stationary environments.Comment: Accepted by KDD 2023. See the project page at
https://xpai.github.io/ReLoo
- …